The present invention relates to semiconductor integrated circuits, and more particularly to a semiconductor integrated circuit which is well suited to form a neural network model in the shape of a monolithic IC.
A neuron consists of a soma, a dendrite and an axon. Of them, the soma generates a pulsing voltage output when its internal potential has risen to exceed a certain threshold value. The pulse voltage is transmitted through the axon until it reaches the distal end of a nerve. The nerve end lies in contact with the dendrite or soma of another neuron. The point of the contact is called a "synapse". The transmission of information between the neurons is effected through the synapse. Such synapses are classified into two sorts; a synapse which has the property of raising the internal potential of the soma of the opposite neuron (hereinbelow, expressed as an "excitatory synapse"), and a synapse which has the property of lowering the internal potential of the soma of the opposite neuron (hereinbelow, expressed as an "inhibitory synapse"). A plurality of synapses are connected to one neuron, and when the pulse reaches one excitatory synapse, the internal potential is caused to rise and then gradually fall in accordance with a certain time constant by the synapse When the pulses reach one synapse continuously, their potentials are successively added with the lapse of time. On that occasion, the arriving pulses have plus or minus weights assigned thereto, depending upon whether they are the excitatory or inhibitory synapses. Therefore, the value of the internal potential becomes the weighted sum of all the inputs. When this value exceeds the threshold value, the neuron generates the pulsing output.
Information processing within the brains of human beings etc. is conducted by a network which is constructed with such a neuron as a unit. Heretofore, in order to explicate the algorithm of the information processing within the brains, researches have been made by modeling the neural network in hardware or software implementation and by performing various aspects of information processing.
Meanwhile, with the progress of semiconductor integrated circuits, researches for realizing the neural network models on semiconductor chips have recently been made vigorously. They are detailed in, for example, "Researches on and Developments of Neurocomputers", NIKKEI ELECTRONICS (dated Jan. 26, 1987, pp. 159-170). The "neurocomputer" mentioned above signifies a computer which is suitable for parallel processing imitative of the operations of the brain of the human being. That is, it achieves the high-speed processing of pattern recognition, combinatorial problems, etc. while utilizing the parallel processing function which is one of the features of the neural network.
FIGS. 7(a) and 7(b) are a symbolic diagram of a neuron, and a graph showing the input/output relationship of the neuron, respectively.
In general, a neural network model is constructed of a plurality of unit circuits, each having many inputs and one output as shown n FIG. 7(a). Hereinbelow, such a unit circuit will be referred to as a "neuron". The respective input terminals of the neuron are given the values of outputs from other neurons. On that occasion, the inputs are multiplied by predetermined weights owing to the functions of the connection parts (hereinbelow, written as "synapses") between the neurons, whereupon the resulting products are afforded to a soma (hereinbelow, written as "cell") 1. FIG. 7(a) shows the symbols of this neuron, and the two sorts of synapses; excitatory synapses 21 having plus weights and inhibitory synapses 22 having minus weights are illustrated.
As stated before, the input/output relationship of the neuron is expressed by the weighted sum of voltages which rise in accordance with a certain time constant. More specifically, as illustrated in FIG. 7(b), the output OUT becomes a function of the summation (.SIGMA..sub.i W.sub.i .multidot.P.sub.i) of the products between all the inputs (P.sub.1, P.sub.2, ..., P.sub.i, ... and P.sub.n) and the weights (W.sub.1, W.sub.2, ..., W.sub.i... and W.sub.n) of the synapses lying at the respectively corresponding input ends.
When a plurality of neurons are coupled to one another and the weights of the individual synapses are appropriately selected, the resultant network can perform significant information processing.
An example of the neural network model as expressed using an electrical circuit is Hopfield's model. In this model, resistors are used for expressing the weights, and gate circuits are used for expressing an input/output relationship similar that of FIG. 7(b).
FIG. 8 is a diagram showing the Hopfield model in the prior art.
This diagram is a model diagram illustrated in U. S. Pat. No. 4660166. In the figure, the mark of a black square indicates a resistor which expresses the synaptic weight between two lines intersecting at the corresponding point, and the marks of a gate and an inverter gate indicate excitatory and inhibitory set inputs and outputs, respectively. The input weights and the connection goals of the plus and minus outputs for three neurons are illustrated in FIG. 8.
In general, in a neural network model, it is sometimes the case that, when synaptic weights are changed on and on in accordance with a predetermined algorithm, they are respectively converged to certain values. Such an operation is usually called "learning".
Several methods have been proposed for the algorithm of the learning. In any case, however, the synaptic weights need to be altered.
In the prior art mentioned above, the synaptic weights are expressed by the resistors, and the corrections of the synaptic weights of a circuit once manufactured require to alter the resistances of the resistors. Therefore, all processes must be redone from the layout of an IC, and the alteration
of a learned content has been difficult.